fix: anchor/alternation PIKEVM routing + B5/B12 backref support by jbachorik · Pull Request #82 · DataDog/java-reggie

jbachorik · 2026-06-18T19:38:57Z

What does this PR do?

Routes anchor-diluted capturing alternations and quantified-group alternation-priority conflicts to PIKEVM_CAPTURE (the priority-correct thread engine) ahead of the DFA paths, with a compileHybrid pre-check and a revert of over-broad PikeVM promotions. Adds backref fixes: B5 (guard lazy quantifiers in VARIABLE_CAPTURE_BACKREF — throw, not silent wrong), B7 (zero-length early-accept for nullable groups), B12 (quantifier-prefix backref bytecode + isPrefixNodeHandleable for unbounded/exact prefixes).

Motivation

Correct captures/matches for pattern classes the DFA strategies mishandled; make lazy-backref gaps fail loudly instead of silently wrong.

Related Issue(s)

Stacked on PR1. Part of the 2026-06 capture-correctness & performance effort.

Improves the safety of (does not close) #33 (alternation-priority decline → fallback) and #37 (B5 lazy-quantifier guard → throws instead of producing wrong captures).

Change Type

Checklist

I have read the CONTRIBUTING.md guidelines
All existing tests pass (./gradlew build)
I have added tests for my changes
I have updated documentation (if applicable)
My commits are signed

Performance Impact

None (routing/correctness).

Additional Notes

Stacked on pr/1-reggieoption-fallback-substrate. Includes a large docs-only commit (dfd070a, plan files). Contains worktree-agent merge commits. Cuts at 1133d2a.

🤖 Generated with Claude Code

Eliminates B10/B15 FallbackPatternDetector predicates and partially eliminates B16 by routing the affected DFA_*_WITH_GROUPS patterns to PIKEVM_CAPTURE before the DFA state-count ladder: - B10: optional prefix before capturing group (e.g. -?(-?.{3}).) - B15: capturing group in quantified alternation (e.g. (a|b){2,}) - B16 (partial): nullable outer quantifier on capturing group with non-nullable content (e.g. (a)?); patterns where both the outer quantifier and group content are nullable (e.g. (0*-?){0,}) still fall back to JDK via the new hasNullableGroupContentWithNullableQuantifier predicate. Both the capture-ambiguous TDFA path and the non-ambiguous DFA-with-groups path now have the three gates before the DFA strategy ladder. Fuzz gate: findings=0 (9530 patterns, 76240 inputs). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add PIKEVM gate inside the capturing TDFA isAnchorConditionDiluted() block: patterns where both branches share a leading character but one branch carries a start-anchor guard (e.g. ^x|x(y)) now route to PIKEVM_CAPTURE instead of the JDK fallback. PikeVM evaluates ^/\A correctly against the search-region origin since commit 0acfc66. Patterns with optional quantifiers, nullable branches, or leading end-anchors still fall through to the anchorConditionDiluted JDK path. Fuzz gate confirms zero divergences with the new routing. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

- NFABytecodeGenerator: add zero-length early-accept before bounds/regionMatches in generateBackreferenceCheck; groupLen==0 trivially succeeds (vacuous match) - FallbackPatternDetector: replace broad hasNullableBackrefGroup B7 guard with narrowed hasAmbiguouslyNullableBackrefGroup that only falls back when the group body can capture strings of length > 1 (unbounded contamination risk); groups with max capture length <= 1 (e.g. a?, [x]?) are safe with the early-accept - BackrefEngineGapsTest: enable b7_nullableBackrefGroupInOptimizedNfa Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…TURE regression)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

- RuntimeCompiler: replace CapturePolicy import with ReggieOption/UnsupportedPatternException - Add cacheKeyFor() helper (flag-aware cache key) and fallbackOrThrow() helper - Gate all 6 JavaRegexFallbackMatcher construction sites behind ALLOW_JDK_FALLBACK flag - compileHybrid() receives ReggieOptions to propagate fallback policy - UnsupportedPatternException propagates through catch(Exception) via explicit re-throw - 34 test files updated: add allowJdkFallback() for patterns requiring JDK fallback - New FallbackPolicyTest: throwsByDefault, delegatesWhenFallbackEnabled, nativePatternUnaffected Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…dings=0

…(Plan A)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…otations ReggieOption moved from reggie-runtime to reggie-annotations so the annotation type can reference it without a circular dependency.

…llback

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…ation

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…ix benchmark

…r-branch check

…onPriorityConflict

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

jbachorik · 2026-06-19T08:23:47Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0b9dc69e9a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copilot

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 3 comments.

)

…nt==0 branch - FallbackPatternDetector.isPrefixNodeHandleable: reject unbounded quantifiers (max==-1); greedy prefix loop commits without backtracking so a*(a+)\1 on "aa" would fail natively. Routes to fallback engine instead. - FallbackPatternDetector.hasStringEndAnchorInAltHelper: unwrap non-capturing groups before the AnchorNode check so (?:\Z)|abc is treated as a pure-anchor branch (same as bare \Z|abc), preventing unnecessary OPTIMIZED_NFA fallback. - PatternAnalyzer: remove dead nfa.getGroupCount()==0 branch inside the nfa.getGroupCount()>0 guard block; zero-group patterns handled outside this block. - Add regression tests for the above in BackrefEngineGapsTest and AnchorAlternationPikeVMTest. - StrategyCorrectnessMetaTest: clarify OPTIMIZED_NFA representative is JDK-fallback. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

datadog-datadog-prod-us1 · 2026-06-19T20:40:14Z

✨ Fix all issues with BitsAI

⚠️ Warnings

🚦 1 Pipeline job failed

CI | build (21)

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: bc24db2 | Docs | Datadog PR Page | Give us feedback!}

jbachorik · 2026-06-19T20:40:59Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a76989652b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…BACKREF Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

jbachorik · 2026-06-19T20:52:28Z

@codex review

jbachorik · 2026-06-19T20:53:25Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4b617a7a1f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

jbachorik · 2026-06-19T21:07:41Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 178413f070

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

jbachorik · 2026-06-19T21:21:42Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c02d9bfa3d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

jbachorik · 2026-06-19T21:32:22Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e9fda8c64c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

All bytecode generators now handle the full Java \$/\Z terminator set: lone \n (with CRLF guard), lone \r, \r\n pair at end-2, NEL, LS, PS.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 99d6ed726e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

jbachorik · 2026-06-19T22:33:26Z

@codex review

…ce; fix enforced gate budget default

jbachorik and others added 30 commits June 11, 2026 17:28

Merge branch 'worktree-agent-a4fe6a65350e71fc7'

ee282fc

merge: resolve PatternAnalyzer conflict between T2 and T3 PIKEVM gates

1735260

fix: extend emitPrefixMatch to handle non-capturing group prefixes (B12)

673df90

Merge branch 'worktree-agent-a4b8f9b78a148b2ce'

7bd6fdd

Merge branch 'worktree-agent-a92292233bc1c2924'

a61fcdd

fix: convert xmlTags benchmark pattern to runtime compile (PIKEVM_CAP…

ef3880a

…TURE regression)

feat: add ReggieOption flag enum

745ef13

feat: replace CapturePolicy with EnumSet<ReggieOption> in ReggieOptions

fd70aea

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

fix: fuzz oracle uses ALLOW_JDK_FALLBACK to preserve coverage and fin…

e9d4ffc

…dings=0

feat: ReggieOption flag substrate + throw-by-default fallback policy …

d891b8d

…(Plan A)

feat: add compilePikeVm staging entrypoint + name-map codec

0b534f0

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

feat: add options() to @regexPattern; move ReggieOption to reggie-ann…

60407a9

…otations ReggieOption moved from reggie-runtime to reggie-annotations so the annotation type can reference it without a circular dependency.

feat: processor classifies methods native/delegate-pikevm/delegate-fa…

3a89b54

…llback

feat: emit delegating stubs for PIKEVM/fallback @regexPattern methods

6b991ce

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

test: end-to-end delegating-stub processor coverage

85414bb

refactor: author PIKEVM xmlTags via @regexPattern delegating stub

816a709

docs: @regexPattern delegating stubs + fallback policy

09f3c9c

feat: PIKEVM delegating stub + @regexPattern fallback policy (Plan B)

ba1e5b8

docs: restore Wave 4 fallback inventory + merge Plan B AGENTS update

5cb7555

test: spike tests for anchor-alternation PIKEVM routing guard classes

fc4eac2

fix: remove over-conservative PIKEVM guards for anchor-diluted altern…

fdd161f

…ation

fix: skip hybrid when DFA anchor-diluted; route to NFA-only path

713f9fd

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

fix: revert unsafe PIKEVM promotions; keep compileHybrid pre-check; f…

cae38f1

…ix benchmark

fix: anchor-alternation PIKEVM guard removal + compileHybrid pre-check

496b56d

fix: route \Z-in-alternation to PIKEVM; narrow FallbackDetector ancho…

768263d

…r-branch check

fix: route anchor+simple-group alternation to PIKEVM before alternati…

9dd4703

…onPriorityConflict

fix: atomic prefix loop + nullable guard + stale comments + rm plan docs

0b9dc69

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

jbachorik requested a review from Copilot June 19, 2026 08:23

Copilot started reviewing on behalf of jbachorik June 19, 2026 08:24 View session

chatgpt-codex-connector Bot reviewed Jun 19, 2026

View reviewed changes

Comment thread reggie-codegen/src/main/java/com/datadoghq/reggie/codegen/analysis/FallbackPatternDetector.java

Copilot AI reviewed Jun 19, 2026

View reviewed changes

jbachorik and others added 2 commits June 19, 2026 22:19

fix: capture correctness across DFA/MGG/GREEDY/RECURSIVE strategies (#83

56e3b89

)

chatgpt-codex-connector Bot reviewed Jun 19, 2026

View reviewed changes

Comment thread ...ie-integration-tests/src/test/java/com/datadoghq/reggie/integration/AlgorithmicFuzzTest.java Outdated

Comment thread reggie-codegen/src/main/java/com/datadoghq/reggie/codegen/analysis/FallbackPatternDetector.java Outdated

fix: disjoint-charset guard for unbounded prefix in VARIABLE_CAPTURE_…

4b617a7

…BACKREF Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

chatgpt-codex-connector Bot reviewed Jun 19, 2026

View reviewed changes

Comment thread reggie-runtime/src/main/java/com/datadoghq/reggie/runtime/PikeVMMatcher.java

fix: clamp negative findFrom start in DFA switch generator

178413f

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

chatgpt-codex-connector Bot reviewed Jun 19, 2026

View reviewed changes

Comment thread reggie-runtime/src/main/java/com/datadoghq/reggie/runtime/PikeVMMatcher.java Outdated

fix: reject $ between \r and \n in PikeVM and MGG anchor check

c02d9bf

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

chatgpt-codex-connector Bot reviewed Jun 19, 2026

View reviewed changes

Comment thread reggie-runtime/src/main/java/com/datadoghq/reggie/runtime/RuntimeCompiler.java

Comment thread reggie-runtime/src/main/java/com/datadoghq/reggie/runtime/PikeVMMatcher.java

fix: extend end-anchor to NEL/LS/PS line terminators in PikeVM and MGG

e9fda8c

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

chatgpt-codex-connector Bot reviewed Jun 19, 2026

View reviewed changes

Comment thread ...en/src/main/java/com/datadoghq/reggie/codegen/codegen/RecursiveDescentBytecodeGenerator.java Outdated

Comment thread ...ie-integration-tests/src/test/java/com/datadoghq/reggie/integration/AlgorithmicFuzzTest.java Outdated

fix: CRLF guard + full terminator set for \$/\\Z across all generators

99d6ed7

All bytecode generators now handle the full Java \$/\Z terminator set: lone \n (with CRLF guard), lone \r, \r\n pair at end-2, NEL, LS, PS.

chatgpt-codex-connector Bot reviewed Jun 19, 2026

View reviewed changes

Comment thread reggie-codegen/src/main/java/com/datadoghq/reggie/codegen/codegen/OnePassBytecodeGenerator.java

refactor: rename zeroDivergenceGate→divergenceGate, enforceZero→enfor…

bc24db2

…ce; fix enforced gate budget default

jbachorik merged commit 21b9bc2 into main Jun 19, 2026
6 of 8 checks passed

jbachorik deleted the pr/2-anchor-alt-routing-backref branch June 19, 2026 23:06

Conversation

jbachorik commented Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Motivation

Related Issue(s)

Change Type

Checklist

Performance Impact

Additional Notes

Uh oh!

jbachorik commented Jun 19, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

datadog-datadog-prod-us1 Bot commented Jun 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Warnings

Uh oh!

jbachorik commented Jun 19, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

jbachorik commented Jun 19, 2026

Uh oh!

jbachorik commented Jun 19, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

jbachorik commented Jun 19, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

jbachorik commented Jun 19, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

jbachorik commented Jun 19, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

jbachorik commented Jun 19, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

jbachorik commented Jun 18, 2026 •

edited

Loading

datadog-datadog-prod-us1 Bot commented Jun 19, 2026 •

edited

Loading